Using multi-GPUs in compute canada

Multi-gpu Interactive Job and Using Jupyter

source

1. ssh to compute canada from WSL terminal ssh vannary@cedar.computecanada.ca

2. Create virtual env in cc.

   1. module load python/3.8
   2. virtualenv <Virtual Env Name>

3. Activate your vitual env.

   1. source <Virtual Env Name>/bin/activate

4. Install Jupyter on Compute Canada.

   1. pip install notebook
   2. pip install jupyterlab

5. Create a wrapper script to launch jupyter notebooks or lab.

   1. echo -e '#!/bin/bash\nunset XDG_RUNTIME_DIR\njupyter notebook --ip $(hostname -f) --no-browser' > $VIRTUAL_ENV/bin/notebook.sh
   2. chmod u+x $VIRTUAL_ENV/bin/notebook.sh
   3. echo -e '#!/bin/bash\nunset XDG_RUNTIME_DIR\njupyter lab --ip $(hostname -f) --no-browser' > $VIRTUAL_ENV/bin/lab.sh
   4. chmod u+x $VIRTUAL_ENV/bin/lab.sh

6. Submit an interactive job.

   1. salloc --gpus-per-node=v100l:4 --time=1:00:00 --ntasks=1 --mem=10G --nodes=1

7. Run the notebook.sh or lab.sh to start a notebook or JupyterLab.

   1. $VIRTUAL_ENV/bin/notebook.sh

   2. $VIRTUAL_ENV/bin/lab.sh

      http://cdr544.int.cedar.computecanada.ca:8888/?token=7ed7059fad64446f837567e3 └────────────────┬───────────────────┘ └──────────┬───────────┘ hostname:port token

8. On new wsl terminal, run ssh -L 8888:<hostname:port> your-username@cedar.computecanada.ca

9. Go to your web browser, type localhost:8888/?token=<token>

10. To watch GPU memory allocation in real time, type watch -n 1 nvidia-smi in the terminal.

Testing Multi-gpus with Fastai in Jupyter notebook

1. Install the following library inside your virtual env (need to be python 3.8 or greater)
   1. pip install torch
   2. pip install fastai
   3. pip install wandb
   4. pip install pathlib
   5. pip install accelerate